19 research outputs found
Optimizing Over Radial Kernels on Compact Manifolds
We tackle the problem of optimizing over all possible positive definite
radial kernels on Riemannian manifolds for classification. Kernel methods on
Riemannian manifolds have recently become increasingly popular in computer
vision. However, the number of known positive definite kernels on manifolds
remain very limited. Furthermore, most kernels typically depend on at least one
parameter that needs to be tuned for the problem at hand. A poor choice of
kernel, or of parameter value, may yield significant performance drop-off.
Here, we show that positive definite radial kernels on the unit n-sphere, the
Grassmann manifold and Kendall's shape manifold can be expressed in a simple
form whose parameters can be automatically optimized within a support vector
machine framework. We demonstrate the benefits of our kernel learning algorithm
on object, face, action and shape recognition.Comment: Published in CVPR 201
Conditional Random Fields as Recurrent Neural Networks
Pixel-level labelling tasks, such as semantic segmentation, play a central
role in image understanding. Recent approaches have attempted to harness the
capabilities of deep learning techniques for image recognition to tackle
pixel-level labelling tasks. One central issue in this methodology is the
limited capacity of deep learning techniques to delineate visual objects. To
solve this problem, we introduce a new form of convolutional neural network
that combines the strengths of Convolutional Neural Networks (CNNs) and
Conditional Random Fields (CRFs)-based probabilistic graphical modelling. To
this end, we formulate mean-field approximate inference for the Conditional
Random Fields with Gaussian pairwise potentials as Recurrent Neural Networks.
This network, called CRF-RNN, is then plugged in as a part of a CNN to obtain a
deep network that has desirable properties of both CNNs and CRFs. Importantly,
our system fully integrates CRF modelling with CNNs, making it possible to
train the whole deep network end-to-end with the usual back-propagation
algorithm, avoiding offline post-processing methods for object delineation. We
apply the proposed method to the problem of semantic image segmentation,
obtaining top results on the challenging Pascal VOC 2012 segmentation
benchmark.Comment: This paper is published in IEEE ICCV 201
SPEGTI: Structured Prediction for Efficient Generative Text-to-Image Models
Modern text-to-image generation models produce high-quality images that are
both photorealistic and faithful to the text prompts. However, this quality
comes at significant computational cost: nearly all of these models are
iterative and require running inference multiple times with large models. This
iterative process is needed to ensure that different regions of the image are
not only aligned with the text prompt, but also compatible with each other. In
this work, we propose a light-weight approach to achieving this compatibility
between different regions of an image, using a Markov Random Field (MRF) model.
This method is shown to work in conjunction with the recently proposed Muse
model. The MRF encodes the compatibility among image tokens at different
spatial locations and enables us to significantly reduce the required number of
Muse prediction steps. Inference with the MRF is significantly cheaper, and its
parameters can be quickly learned through back-propagation by modeling MRF
inference as a differentiable neural-network layer. Our full model, SPEGTI,
uses this proposed MRF model to speed up Muse by 1.5X with no loss in output
image quality
EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval
Large neural models (such as Transformers) achieve state-of-the-art
performance for information retrieval (IR). In this paper, we aim to improve
distillation methods that pave the way for the resource-efficient deployment of
such models in practice. Inspired by our theoretical analysis of the
teacher-student generalization gap for IR models, we propose a novel
distillation approach that leverages the relative geometry among queries and
documents learned by the large teacher model. Unlike existing teacher
score-based distillation methods, our proposed approach employs embedding
matching tasks to provide a stronger signal to align the representations of the
teacher and student models. In addition, it utilizes query generation to
explore the data manifold to reduce the discrepancies between the student and
the teacher where training data is sparse. Furthermore, our analysis also
motivates novel asymmetric architectures for student models which realizes
better embedding alignment without increasing online inference cost. On
standard benchmarks like MSMARCO, we show that our approach successfully
distills from both dual-encoder (DE) and cross-encoder (CE) teacher models to
1/10th size asymmetric students that can retain 95-97% of the teacher
performance
A Framework for Shape Analysis via Hilbert Space Embedding
We propose a framework for 2D shape analysis using positive definite kernels defined on Kendall’s shape mani-fold. Different representations of 2D shapes are known to generate different nonlinear spaces. Due to the nonlinear-ity of these spaces, most existing shape classification algo-rithms resort to nearest neighbor methods and to learning distances on shape spaces. Here, we propose to map shapes on Kendall’s shape manifold to a high dimensional Hilbert space where Euclidean geometry applies. To this end, we introduce a kernel on this manifold that permits such a map-ping, and prove its positive definiteness. This kernel lets us extend kernel-based algorithms developed for Euclidean spaces, such as SVM, MKL and kernel PCA, to the shape manifold. We demonstrate the benefits of our approach over the state-of-the-art methods on shape classification, cluster-ing and retrieval. 1
Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices
Symmetric Positive Definite (SPD) matrices have become popular to encode image information. Accounting for the geometry of the Riemannian manifold of SPD matrices has proven key to the success of many algorithms. However, most existing methods only approximate the true shape of the manifold locally by its tangent plane. In this paper, inspired by kernel methods, we propose to map SPD matrices to a high dimensional Hilbert space where Euclidean geometry applies. To encode the geometry of the manifold in the mapping, we introduce a family of provably positive definite kernels on the Riemannian manifold of SPD matrices. These kernels are derived from the Gaussian kernel, but exploit different metrics on the manifold. This lets us extend kernel-based algorithms developed for Euclidean spaces, such as SVM and kernel PCA, to the Riemannian manifold of SPD matrices. We demonstrate the benefits of our approach on the problems of pedestrian detection, object categorization, texture analysis, 2D motion segmentation and Diffusion Tensor Imaging (DTI) segmentation. 1